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We give a solution to the problem of designing a fixed compromise 
equalizer for use in transmission systems involving an ensemble of random 
channels. The signal and noise spectra, along with the second-order 
statistics of the channel ensemble, are used to find the equalizer charac- 
teristic that minimizes the mean-square distortion between the equalizer 
output and a scaled version of the transmitter output. The key departure 
from previous work is that the criterion better captures practical per- 
formance invariance; specifically, the cost function incorporates the 
insensitivity of a well-designed demodulator to any amplitude scaling or 
time delay introduced by a particular channel. After demonstrating that 
the optimum equalizer shape is related to the principal eigenf unction of a 
normalized channel correlation function, we consider several special cases 
that give further insight into the properties of the solution. We find that 
the equalizer amplitude is attenuated over those frequencies where the 
signal-to-noise or signal-to-channel-variance ratios are small. The analysis 
confirms the standard engineering practice of inverting the average channel 
in the absence of noise and when the variance of the channel characteristics 
is small. 

I. INTRODUCTION 

A fixed compromise equalizer is frequently employed in data trans- 
mission systems to compensate for linear distortion introduced by a 
channel drawn from a random ensemble. 1 Typically, compromise 
equalizers find application in systems which, because of economic or 
other considerations, do not use an adaptive equalizer. It is possible 
that, even when adaptive equalization is used to compensate for a 
particular channel characteristic, one might use a compromise equalizer 
to provide a good initial channel and thereby reduce the receiver 
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adaptation time. As its name suggests, the equalizer is a fixed linear 
filter that effects a compromise by compensating for an "average" 
channel. We propose a procedure that uses the statistics of the channel 
ensemble, the modulated signal, and the additive noise at the demodu- 
lator to design a filter that minimizes a performance measure ap- 
propriate for most transmission systems. The performance measure is 
an adaptively scaled mean-square error. For example, it is particularly 
well suited for use in a data transmission system and results in a filter 
that is significantly different from that obtained by directly minimizing 
the mean-square error. 2 

In order to avoid being restricted by nonlinear demodulation tech- 
niques (e.g., those used in FSK or DPSK data systems) and to be 
able to accommodate asynchronous signalling, the equalizer operates 
directly on the received passband signal and is thus a channel, rather 
than a synchronous, equalizer. Our performance criterion is the con- 
tinuous time mean-square error between the equalizer output and an 
adaptively scaled version of the transmitter output. This scaling needs 
to be done only once in the design of the filter and not each time a new 
channel is dialed up. The scaling is such that the filter is invariant to 
any amplitude scaling, sign inversion, or time delay encountered in 
transmission over a particular channel; this type of invariance is 
appropriate for a compromise equalizer used in most transmission 
systems, since amplitude scaling and a fixed time delay will not be 
"seen" by a well-designed demodulator. 

Since the criterion is quadratic in nature, the optimum filter re- 
sponse is determined by the second-order statistics of the signal, noise, 
and channel ensemble. The best filter shape is shown to be the principal 
eigenfunction of an integral operator whose kernel is a weighted channel 
correlation function. An explicit design procedure is described in the 
text, and several interesting questions are discussed as well. Some of 
these questions are: 

(i) How different is the compromise equalizer from the inverse of 
the average channel characteristic? 

(ft) Suppose there is no amplitude distortion but only delay distor- 
tion. What is the nature of the compensation? Is there amplitude 
as well as delay compensation? 

(in) How sensitive is the filter design to different signal spectra? 

In Section II the compromise equalization problem is formulated, 
and the distortion measure is discussed in considerable detail. The 
determination of the optimum filter is described in Section III, and 
an example is provided in Section IV illustrating the design technique. 
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Fig- 1 — Transmission system with a compromise equalizer. 



II. PROBLEM FORMULATION 

2.1 System Configuration 

We begin by specifying the framework of our discussion. To accom- 
modate a system employing nonlinear demodulation, the compromise 
equalizer is assumed to be placed in the passband. Since we also want 
to provide a setting general enough to include asynchronous transmis- 
sion, no restriction is placed on the form of the filter other than require- 
ment of a finite energy impulse response. As shown in Fig. 1, s(t) 
denotes the modulated signal, n(t) the received additive noise, x(t) the 
compromise equalizer output, and g(t) the equalizer impulse response 
(which we ultimately seek to specify). The ensemble of channel impulse 
responses, \h{t)), as well as n(t) and s(t), are all assumed to be con- 
tinuous in quadratic mean and statistically independent of each other. 
Both s(t) and n(t) will be taken to be zero mean, to be wide-sense 
stationary with finite power and to possess power spectral densities 
S(u) and iV(aj), respectively. To make the subsequent analysis precise, 
we assume h(t) has bounded absolutely integrable sample paths (with 
probability one); thus, the ensemble of channel frequency charac- 
teristics \H(io)} is well defined. + 

Our problem is first to select an appropriate optimization criterion 
and then to choose the compromise equalizer that minimizes this 
measure. To avoid the specifics of particular demodulators, we will be 
concerned with preserving the fidelity between s(t) and x(t). A natural 
first choice for a distortion measure is the continuous-time mean- 
square error 



S = <[>(0 - s(t - r)]»), 



(1) 



where < ) denotes the average with respect to the signal, noise, and 
channel ensemble, and t is an arbitrary delay. Using this measure, 
Maurer and Franks 2 found that the optimum filter is given by 



UoW S( w )<|ff(«)|«) + tf(«) ' 



(2) 



* The process {#(«)) is assumed to have a square integrable covariance, and the 
variance is assumed to be nonzero for the frequency range of interest. 
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where the asterisk denotes the complex conjugate and H (o>) is a member 
of the channel ensemble. It is easy to see that G (a), given by (2), is 
not just the Wiener filter for the average channel, since the mean- 
square channel dispersion <|#(w)| 2 ), rather than |(#(a>))| 2 , modifies 
the filter characteristic. We note that, even as N(u>) — > 0, the filter does 
not generally invert the average channel. This will be the case when 
the channel variance is extremely small so that ( | H(u) | 2 ) tt \ (H(w)) | 2 ; 
this would occur, for instance, when the ensemble consists of only one 
characteristic. 

Care must be exercised in determining how to use the design tech- 
nique that results in the filter given by (2). For example, suppose the 
individual transfer characteristics are greatly varied in the degree of 
attenuation they impart across the band of interest. It can follow 
(depending on how probabilities are attached) that the low-loss chan- 
nels determine the character of the averages in (2) — the "whomper" 
effect. Hence, linear distortion in the lossy channels may not be 
suitably equalized. However, if linear distortion is uniformly the 
dominant impairment, the noise level can be set to zero and the 
"whomper" effect easily eliminated by preparing the data by normaliz- 
ing the channels prior to averaging. For example, the normalization can 
be accomplished by individually scaling the characteristics so they 
have the same energy in response to some pulse or so they have the 
same gain at some central frequency. Even if linear distortion is not 
the uniformly dominant impairment, there may be applications where 
the noise is set to zero and a normalization of the channels made prior 
to computing the optimum equalizer. In such a procedure, one is trad- 
ing noise immunity for immunity to linear distortion. Another case of 
importance in applications is when the channels have the same ampli- 
tude characteristic but different phase characteristics ; here, of course, 
the "whomper" effect is nonexistent. The considerations of this para- 
graph will also apply to the design technique we shall develop in the 
following sections. 

While the 8 criterion provides an interesting and tractable formula- 
tion, it has the shortcoming that it understates the capability of most 
demodulators. A striking example of this occurs if the ensemble { h(t) } 
has zero mean. The zero mean is reasonable for systems subject to 
occasional "phase hits," the mathematical implication being that, if 
h(t) is a possible channel, then —h(t) is just as probable. Then G (co) = 0. 
Yet, in practice, one would expect to do better than "pull out the plug 
and go home." 

At the other extreme we could use an information theoretic type 
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criterion. For example, we could choose G(u) to maximize the average 
mutual information between x(t) and s(t). This criterion is mathe- 
matically tractable; in fact, the solution is trivial — G(u) can be any 
characteristic which is nonzero over the same frequency range as 
#(«)•* The shortcoming of this criterion is (as the solution suggests) 
that it overestimates the demodulation capability. 

As the state of the art in demodulation advances, we would antici- 
pate an evolution of criteria away from the mean-square error toward 
the information theoretic. Clearly, a good criterion should give the 
demodulator credit for what it can realistically accomplish and at the 
same time pose a tractable optimization problem for determining G'(o>) . 

With this motivation, we propose a mean-square-error criterion 
which reflects the fact that the performance of a well-designed demodu- 
lator is insensitive to scaling of the input (the automatic gain control 
feature), an input sign change (the information is generally differ- 
entially encoded), and, of course, a time delay. Under such a criterion, 
the compromise equalizer will no longer be implicitly constrained by 
attempting to faithfully reproduce the modulated signal. 

2.2 An Adoptively Scaled Mean-Square Error 

Based upon the above discussion, we consider the following adapt- 
ively scaled mean-square error 

8 = min ([_x(t) ~ As(t - £)] 2 > 8 .„, (3) 

A,B 

where A and B are real numbers and the averaging is over the signal 
and noise statistics with both the channel and equalizer held fixed. 
The criterion is meaningful under the assumption that the signal-to- 
noise ratio at the receiver does not change appreciably from channel 
to channel. We stress that the considerations mentioned in the third 
paragraph of Section 2.1 apply here, as well. The optimum equalizer 
is obtained by averaging S over the channel ensemble and then 
minimizing the result with respect to the equalizer transfer function 
(7(oj), subject to a power constraint on the demodulator input. The 
quantities A and B, which are channel-dependent, provide an adap- 
tively scaled reference signal As(t — B). The reference is adaptive in 
that, for each realization of the channel, A and B are chosen to mini- 
mize 8. Notice, for example, that, if a particular channel introduces a 
sign inversion, then A = — 1 will remove this effect. When a channel 



+ This comes about because no information is lost when the signal is subject to a 
reversible operation (such as a channel). 
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has a more complicated phase and/or gain characteristic, it is no 
longer apparent what the optimal value of A should be; however, we 
shall shortly see that the minimizing value of A can be determined 
analytically. We shall also consider the determination of B. Thus, the 
filter will not expend any of its degrees of freedom by attempting to 
compensate for a sign inversion, amplitude scaling, or time delay 
introduced during transmission. It should be clear that, since A and 
B depend on the channel characteristics, they are random variables. 
Simply put, the criterion given by (3) forces the equalizer to minimize 
only the portion of the output signal that does not look like a scaled or 
delayed version of the transmitted signal. 

We now consider the properties of the adaptively scaled mean- 
square error. We begin by letting 

/ - (lx(t) - As(t - B)?>.,», (4) 

i.e., 

S = min I. 

A,B 

Carrying out the indicated average gives 

/= r {S(co)[|F(a ) )| 2 -2Ae^F( w )+A 2 ]+ | G(«) 1 2 N(o>) } ^ , (5) 

J— co 

where £(a>) is the power spectral density of s(t) and F(u) is the product 
of G(u) and #(u>). (Notice / does not change when F(u) is replaced by 
Re{F(u))\.) To find the minimum of / with respect to A and B, we set 
to zero the partial derivatives of / with respect to these variables and 
find that 

£L =-2J" S(«)e*"F(«) ^ + 2A £° 5(«) ^ = (6a) 

A[ = -2J^ ^(.)i*VM £ = 0. (6b) 

From (6a) we have A opt given by the correlation ratio 

gsM^m^ (s(t B)xm 

= />•>£ = <s,(( " B)} ' 

The interpretation of B opt is facilitated by letting 

y(t) mf m j*F(*)B{*)+«ft, (8) 
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Fig. 2 — Interpretation of optimum delay (B op t) and amplitude (A ov t) scaling. 
B op t is one of the instants when y{t) is zero. 

which is recognized as the response of a linear filter with transfer 
function juF(w) [i.e., F(<o) followed by a differentiator] to the input 
R(t), where R(t) is the (inverse) Fourier transform of S(w) and is the 
signal correlation function. Comparing (6b) and (8), we see that B opt 
is one of the instants when y(t) is zero. We illustrate this interpretation 
of .Bopt in Fig. 2, where we also indicate how A opt may be obtained in 
a similar manner. Determining B opt is a very difficult problem, since 
it is tantamount to asking for the zero crossing of a signal from knowl- 
edge of its Fourier transform. In order to proceed further, we will 
approximate B opt by the delay at midband, which we conveniently take 
to be zero for each channel. 

Using the value of A opt given by (7) and setting B = 0, we have 

s ■ /_! ' G(<i}) ' 2[<s(w) i H(a}) ■ 2 + N(u) l 2^ 



, 0) 



where 






The optimum filter is obtained by first averaging (9) with respect to 
the ensemble of channels and then minimizing this average with 
respect to G(w). + Before doing this, we give geometric interpretations 
to both the criterion and the optimum filter. 

2.3 A Geometric Interpretation of the Problem 

Finding a geometric framework in which to view the optimization 
problem will be quite helpful in understanding the nature of the 
solution. To put the problem at hand in such a setting, we introduce 



f The minimization is done subject to a constraint on the average output power. 
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.1 
GH, 




Fig. 3— Geometric interpretation of criterion. The adaptively scaled mean-square 
error may be interpreted as the average squared distance from the point GH (which 
represents an equalized channel) to the ray 1 (which represents a distortionless 
channel) . 



(10) 



the signal-weighted inner product 

(U, V) s£^t7(«)7*(«)5(«)^, 

where the vectors U and V represent the functions U(a>) and V(w) 
respectively, and the vector 1 will correspond to the real function of 
unit amplitude. Suppose that the noise is set to zero; then, in terms of 
the above notation, we can write 1 



8 = ||GH|| 2 --(GH, l) 2 , 



(11) 



where the vector GH corresponds to the function G(co)H(u). For 
convenience we set the signal power equal to unity, i.e., a = 1, and 
apply the Schwarz inequality, which gives 

8 = ||GH|| 2 - (GH, l) 2 ^ ||GH|| 2 - ||GH|| 2 - ||l|| 2 = 0, (12) 

where the lower bound is achieved when GH is proportional to 1.* 
From (11) we have 

(GH, l) 2 1 



S = (GH, GH) - (GH, l) 2 = (GH, GH) 1 

= ||GH|| 2 [1 - cos 2 0] = [||GH|| sin 0] 2 , 



(GH, GH) 



(13) 



t The norm of the vector U, denoted by ||U||, is given by (U, U)». 
1 Thus, if we have only one channel characteristic and no noise, the equalizer will 
invert the channel. 
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where d is the angle between GH and 1. Equation (13) provides a very 
useful interpretation of the error 8. As shown in Fig. 3, ||GH|| sin 9 is 
the distance from the vector GH to the ray colinear with the vector 1. 
Hence, the average of 8 with respect to the channel ensemble, which we 
denote by (<§) H , is the average squared distance from GH to the ray 1. 
Thus, for a given channel-equalizer constellation, { GH,}, the equalizer 
is chosen to minimize the dispersion about the ray 1. 

In order to get a feeling for the capability of the equalizer to modify 
the channel constellation j H, } , we associate with G'(o>) a linear operator 
9 that maps a particular channel H, into GH,; we write this operation 
symbolically as 

9:H,- >GH,. (14) 

The equalizer operator, g, is called diagonal since it modifies H(u>) in 
a pointwise fashion to produce G(co)H(o)). Suppose, for the purpose of 
illustration, we relax our hypothesis and assume that the channel 
ensemble has energy only at two values of a», cc t and o> 2 . In this case, the 
operator g is particularly simple since the point H = (hi, h 2 ) is mapped 
into the point GH = (gih u g 2 h 2 ), where /t, = H(m) and g { = G'(co,). 
The locus of points (gihi, gih 2 ), subject to the average power constraint 
on the demodulator input x(t) 

g\ki + gr§fc 2 = 1 (ki ± E{ \H(m) 1 2 SM }), (15) 

describes the manner in which the equalizer redistributes the channel 
ensemble so as to minimize the average squared distance to the ray 1. 
By noting that the point (gihi, g 2 h 2 ) satisfies the relation 

~wr + ~wr = l > (16) 

we see that the locus of the points Mi, g 2 h 2 ) subject to (15) is an ellipse 
centered around the origin whose major and minor axes are parallel to 
the x-y axes and are of length [ h\/k\ \ and | h 2 /k\ \ . Thus, the effect 
of the equalizer on the channel array, as shown in Fig. 4, is to allow 
each equalized channel, GH, = [ghfi, ghjp], to move on an elliptical 
surface. The nature of the compromise by which the optimum filter 
shape is chosen should now be clear. Each channel wants to move as 
close to the unit ray as possible — this will usually lead to conflicting 
requirements, i.e., as one channel is brought closer to the unit ray, 
other channels will move further away from this ray. For a multiplicity 
of channels there will be an "elliptical flow" along the respective 
ellipses which terminates when the best filter shape has been found. 
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Fig. 4 — Geometric interpretation of the effect of the compromise equalizer on the 
channel ensemble. A channel, represented by the point H„ is modified by the equalizer 
to give the equalized channel GH ; . 

The above interpretation would of course be valid in a Hilbert space 
of dimension large enough to accurately represent the function i/(«). 

III. DETERMINING THE OPTIMUM EQUALIZER CHARACTERISTIC 

3.1 Analytic Solution 

Having developed a geometric representation as an aid to under- 
standing the equalizer design problem, we are now in a position to 
explicitly determine the best filter shape. Returning to (9) and averag- 
ing with respect to the channel ensemble, we have 

( S ) H = r ig( W ) i»[iS(«)iaL(«) + #(«)] ^ 



where 
If we let 



" \ F I- WWW^^W^W) ^ 5F » (17) 



ff,m.(«) " V(|H(0,)| 2 >. 



Q(«) = VS( u )flL(w) + JV(cu)G(«) 
and introduce the weighted channel covariance kernel 
1 S(a)S(y)(H*(o)H(r)) 



K(t>, v) = 



« V[5(«)^™(«) + ^(«)][5(v)fOLW + N{v)\ ' 



(18) 
(19) t 

(20) 



t Note that \Q(u) \ 2 is the power spectral density of the equalizer output for the 
"rms channel." 
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then our cost function can be rewritten in the convenient form 

(&)h = JT Q(„)Q*(«) g - £ £ £(«, ,)<?*(«)Q(.) g £ (21) 

Introducing the Hermitian integral operator 3C, whose kernel is K(w, v), 
permits us to write the criterion as the quadratic form 

(S)h = (Q, Q) - (JCQ, Q), (22) 

where the inner product does not include the signal-spectrum weighting 
introduced in (10). f We now wish to minimize (22) with respect to Q, 
subject to an appropriate constraint. Referring to Fig. 1, we see that 
the average power present at the demodulator input is 

P-jT D8(-)JHUW +iV(o»)]|G( w )| 2 ^- 

Thus, a natural constraint is to require that the power be constant. 
In terms of Q(u), this constraint takes the form 

(Q, Q) = P, (23) 

and the optimization problem consists of minimizing the positive 
definite form (22) subject to (23). The solution, which we denote by 
Qopt, is easily obtained 3 by using a Lagrange multiplier and is recog- 
nized as the principal eigenfunction* of the operator 3C. The best filter 
shape, Cr 0P t(w), is obtained from Q op t(w) by using (19), and can be re- 
garded as the first term in a Karhunen-Loeve representation 4 of the 
random process 

M = H(»). (24) 

The residual value of the criterion, evaluated when Q = Q 0P t, is 
given by 

<£>opt = (Qopt, Qop.)[l - X] = (1 - X)P, (25) 

where X is the maximum eigenvalue and is a measure of how effectively 
the compromise equalizer performs for various channel ensembles and 
various signal and noise spectra. The degree to which the random 
process given by (24) is described by the first term in a Karhunen- 
Loeve expansion will, of course, determine the equalizer performance. 

* In the sequel, the inner product between the vector U and V will be taken to be 
(U . V) = f* U(f?)VM (d»/2ir) . 

» That is, the eigenfunction corresponding to the maximum eigenvalue (it is well 
known that the eigenvalues of a Hermitian operator are real). 
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In order to get more insight into the nature of G op t(w) we will, in the 
next paragraph, consider the form of the optimum filter under some 
special circumstances. This discussion, along with the examples treated 
in the next section, will reveal some properties of the optimum filter 
shape. 

3.2 Special Cases 

3.2.1 One Channel 

Suppose the channel ensemble consists of only one member,* H(w). 
The principal eigenfunction of 3C is easily determined to be 

*-w - jwm^wrm \ H * (w) - (26) 

\5(u) \H{o)) | 2 + iV(a>) 

Thus, the best filter shape is 

S(co)H*(u>) (2?) 

(7opt(w) ~5(co)|i/( W )| 2 + iV(a,) * ' 

The filter given by (27) is just the well-known Wiener filter for estimat- 
ing a random signal that has been passed through a nonrandom 
channel and then corrupted by additive noise. The amplitude charac- 
teristic of the filter, which is given by 

S(u) \H(<a)\ / 2 g) 

S(a,)|#(a;)| 2 -r- AT(cu)' 

provides noise rejection at those frequencies where S(u) is small relative 
to N(u). Thus, as the spectrum of the modulating signal is changed, 
the noise-rejecting regions of the filter will be altered. It is worth 
noting that, if the channel has only phase distortion, amplitude as well 
as phase compensation is required. 1 It should also be noted that the 
amplitude response of the filter will be greatly attenuated at those 
frequencies where |tf(o>)| 2 dominates JV («)/£(«); this phenomenon, 
which we call channel-variance rejection, is observed in practice for an 
arbitrary channel ensemble at the frequencies where the variance of 
H(u), defined by 

Var[#(co)] ■ <|H(«)| 2 ) - |<#(o>))| 2 , (29) 

becomes large. Since the phase of G opt (a>), for the simple case of one 
channel, is just the negative of the channel phase, we have, under the 
further specialization of vanishingly small noise power, the not- 

* This assumption gives good insight when there is small dispersion about the 
average channel, i.e., the channels all look pretty much alike. 

* This, along with the preceding sentence, provides a partial answer to the second 
and third questions posed in the introduction. 
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surprising result that the equalizer should invert the channel. On the 
other hand, as the noise becomes dominant, the solution is observed to 
approach a filter matched to the corresponding average channel 
characteristic and signal spectrum. 

3.2.2 Deterministic Amplitude Distortion, Random Delay Distortion, 
and No Noise 

Suppose the members of the channel ensemble can be written as 
H(co) = a(u)e»<»>, (30) 

where the amplitude response a(w) does not vary from channel to 
channel, and the phase response 0(a>) is randomly selected. 

We consider first the kernel K( , ) when there is only delay distortion 
(i.e., a(co) = 1) and no noise. Since H rms (w) is unity, we have 



Xl^j.ffii^H.W), (31) 

and the filter is given by 

Gdelay(«) = %^- (32) 

If we no longer restrict a(a>) to be unity, the kernel is still given by 
(31) while the filter is seen to be 

G(u) = ^yGWAY(w). (33) 

The above indicates that, in the presence of deterministic amplitude 
distortion and random phase distortion, the compensation is decoupled 
in the sense that the filter shape is a cascade of the compensation for 
the component distortions. While this sort of decoupling is generally 
not the case, we have found in the example described in the next 
section that the phase characteristic of the equalizer is rather insensitive 
to changes in amplitude distortion and noise level. 

3.2.3 Small Variation in the Amplitude and Phase Characteristics 

We now wish to give a suggestive description of the optimum filter 
when the channel characteristics have small variation about the 
average characteristic. To this end, let us denote a typical charac- 
teristic by 

H(u) = a(u)e^»> 

= a(Q>)&(«)e''i'<»> + *<">], (34) 
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Fig. 5— (a) amplitude and (b) delay characteristics for average channel. The 
worst case amplitude and phase variation are indicated by the isolated circles. 

where d(o>) and 0(w) are the average channel amplitude and phase, 
respectively, and b(«) andtf>(a>) represent small (random) perturbations 
about these quantities. We first write #(«) as 



where we have let 






(34a) 
(34b) 



H(u) = a(co)e>^ u) (35) 

g(«) = lnb{u) + ;+(«), (36) 

and we note that H(w) is composed of the average amplitude and phase 
of the ensemble. Since |b(«)|«l and |<£(co)|^0, we see that 
| q(a) | « 0; thus, retaining only the leading term in the series expansion 
of e q(u) gives 

e ,( w) ~ i _|_ g ( w ). (37) 
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Fig. 6 — Raised cosine power spectral density (.rolloff = 1). 

We are now in a position to evaluate the kernel K(u>, v) under the 
above assumptions. Using (37) we have 

(H*(u)H(v)) = R*(u)R(»)(e*'i»W>) 

K B*(*)B<?)(1 + q*(co) + q{v)), (38) 

where we have kept only first-order terms in the expansion of e q * {w) e q< -" ) . 
If we assume that the perturbations tnb(u>) and qfr(w) have zero average 
value, then (38) separates, reducing to 

(H*(u)H(p)) = R*(u)R(p). (39) 

By a similar argument, the denominator of the kernel is found to be 



V^M \R(a) I 2 + N(u) X ylS(v)\R(v)\ 2 + N(v), 
and combining (39) and (40) gives the separable kernel 

8((u)S(v)S*(u)R(v) 



K(u, v) = 



VS(co) \R(a) | 2 + A T («)VS(„) |#(„) 1 2 + N(p) 
The equalizer shape is then given by 



G op t(o>) = 



S(a,)|#(co)| 2 + N(u)' 



(40) 



(41) 



(42) 
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Fig. 7 — Equalizer amplitude characteristics for various noise levels. 

which is of the same form as the one-channel result [see (27)], as well 
as the filter obtained by Maurer and Franks (see Ref. 2). Clearly, 
for those frequencies where the noise is negligible_(compared to the 
signal) the equalizer characteristic is Cl/fl(w)> _, '* ( " ) > i.e., the best 
filter is one which inverts the average channel. Recalling the first 
question posed in the introduction, we can see that, apart from noise 
rejection (which occurs in the frequency range of small signal spec- 
trum), the filter will invert the average channel when the variance of 
the ensemble is not appreciable. This is a useful rule-of-thumb for 
rapid (and approximate) compromise equalizer design. 

IV. EXAMPLES USING THE 1964 CUSTOMER LOOP SURVEY 

In this section we consider the design of a compromise equalizer for 
use over an ensemble of data customer loops. Some knowledge of the 
characteristics of this ensemble can be obtained from the 1964 Loop 
Survey. 5 This survey collected information about transmission param- 
eters and channel makeup (e.g., gauges and lengths of sections which 
compromise the loop, locations of load coils and bridged taps, etc.) for 
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Fig. 8 — Average channel delay (inverted) and equalizer delay (SNR = 30 dB). 



a random selection of loops. For our purposes we extracted an ensemble 
of business loops (as opposed to residential loops) described in the 
survey. An existing computer program was used to obtain the fre- 
quency responses of the channels (with load coils and bridged taps 
removed) from the descriptions of the physical makeup of the channels. 
A recent study 6 shows that, at voiceband frequencies, the functions 
[e~ y/iua ) a >a provide a good approximation of the loop transfer charac- 
teristics. The parameter a is a random variable (whose distribution 
can be approximated by the survey information), specifically a = RCl 2 , 
where R is the average series resistance per mile, C is the capacitance 
per mile, and ( is the loop length in miles. The average channel is 
depicted in Figs. 5a and 5b in terms of gain and delay. 1 These loop 
characteristics display appreciable amplitude variation (as well as 
delay variation) and the extent of these variations is indicated by the 
isolated circles. 

To illustrate the design technique on these loop networks, we deter- 
mined a compromise equalizer characteristic to be employed in a 
maxentropic 4.8-kb/s stream of randomly signed pulses. The basic 



+ The channel delays each include an estimate of B opt . 
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Fig. 9 — Reduction of residual error as a result of equalization. 



pulse was assumed to have a raised cosine spectrum, and the resulting 
power spectral density is shown in Fig. 6. It was assumed that only 
the "worst" 70 of the 143 customer loops required equalization. Figure 
7 shows the resulting equalizer amplitudes for various noise levels. In 
this example, the channels were normalized to avoid the "whomper" 
effect mentioned in Section II. So when the signal-to-noise ratio is not 
infinite, the noise levels should be interpreted as if the lowest signal- 
to-noise ratio for the entire ensemble was assumed for all channels. 
Observe that, in the absence of noise, the equalizer tends to invert the av- 
erage channel amplitude characteristic, and as the noise level is increased 
the equalizer tends to attenuate the high end of the spectrum where the 
signal power is lowest. Also note that, as the signal-to-noise ratio de- 
creases, the solution tends to a filter matched to the raised cosine 
characteristic times a characteristic approximating the average 
channel. We observed that, for each of the noise levels, the compromise 
equalizer delay is close to the inverse delay of the average channel. 
The specific delay curve for the case of 30-dB SNR is provided in Fig. 8, 
which also shows the average channel delay (inverted). Finally, a 
direct computation shows that, for typical signal-to-noise ratios, the 
equalizer reduces the residual error by 4.8 dB, and Fig. 9 displays this 
improvement in mean-square error as a function of signal-to-noise 
ratio. The improvement is greatest at high SNR since the equalizer is 
combating only linear distortion rather than a combination of linear 
distortion and noise. 
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VI. CONCLUSIONS 

Using the second-order statistics of the channel ensemble, as well as 
the signal and noise spectra, the equalizer characteristics were easily 
computed by solving a matrix eigenvalue problem. Several interesting 
conclusions can be drawn concerning the properties of the equalizer: 

(i) The equalizer amplitude is attenuated in those frequency 
regions where the signal-to-noise ratio or signal-to-channel- 
variance ratio is small. 
(ii) When the channel ensemble has only delay distortion, ampli- 
tude as well as phase compensation is required. 
(Hi) The delay characteristics of the equalizer are rather insensitive 
to changes in noise level or (non-random) amplitude distortion. 
(iv) When the channel ensemble has small variation about the 
average characteristic, a situation that commonly arises in 
practice, then the equalizer will invert the average channel. 
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